Predicting interior models of structures

ABSTRACT

Methods and systems for improved prediction and generation of structure interiors are provided. In one embodiment a method is provided that includes receiving exterior imagery of the structure and determining an exterior surface of the structure with a machine learning model. The exterior surface may enclose exterior portions of the structure. The machine learning model may further determine exterior features of the structure and may determine, based on the exterior surface of the exterior features, an interior model of the structure. A three-dimensional representation of interior and exterior portions of structure may be generated based on the exterior surface and the interior model.

BACKGROUND

In various situations and applications, it may be advantageous to knowthe interior layout of a structure prior to entering the structure. Forexample, interior layouts may be useful, e.g., in emergency responsescenarios, demolitions analysis, military scenarios, and the like.

SUMMARY

The present disclosure presents new and innovative systems and methodsfor improved prediction and generation of structure interiors. In afirst aspect, a method is provided that includes receiving exteriorimagery of a structure and determining, with a machine learning model,an exterior surface of the structure. The exterior surface may encloseexterior portions of the structure depicted within the exterior imagery.The machine learning model may also determine exterior features of thestructure based on the exterior imagery and/or the exterior surface. Themachine learning model may also determine an interior model of thestructure based on the exterior surface and the exterior features. Themethod may also include generating a three-dimensional representation ofinterior portions of the structure and exterior portions of thestructure based on the exterior surface and the interior model.

In a second aspect according to the first aspect, the exterior featuresinclude at least one of doors, windows, structural support elements,corners, roofs, and/or utility systems of the structure.

In a third aspect according to any of the first and second aspects, thestructure has multiple floors and determining the interior modelcomprises determining multiple interior models for the multiple floors.

In a fourth aspect according to the third aspect, the multiple floors ofthe structure are identified based on exterior features of thestructure.

In a fifth aspect according to the fourth aspect, the multiple floorsare identified based on multiple levels of windows at multiple heightswithin the structure.

In a sixth aspect according to any of the first through fifth aspects,the method further includes, prior to receiving the exterior imagery,receiving training data for a plurality of structures. The training datamay include exterior imagery of the plurality of structures, expectedexterior surfaces for the plurality of structures, expected exteriorfeatures for the plurality of structures, and expected interior modelsfor the plurality of structures. A first machine learning model may betrained to generate predicted exterior surfaces and predicted exteriorfeatures of at least a subset of the plurality of structures based atleast on (i) exterior imagery of the subset of the plurality ofstructures and (ii) expected exterior surfaces of the subset of theplurality of structures. The first machine learning model may also betrained to predict interior models for at least the subset of theplurality of structures based at least on the predicted exteriorsurfaces and the predicted exterior features.

In a seventh aspect according to the sixth aspect, the method furtherincludes, prior to receiving the training data, receiving a plurality ofarchitectural plans for the plurality of structures and generating, witha second machine learning model, the expected interior models for theplurality of structures based on the plurality of architectural plans.

In an eighth aspect according to the sixth and seventh aspects, trainingthe first machine learning model to predict exterior contours andexterior features includes receiving first exterior imagery of a firststructure from the plurality of structures and predicting, with thefirst machine learning model, a first exterior surface of the firststructure and first exterior features based on the first exteriorimagery. One or more differences may be detected between (i) between thefirst exterior surface of the first structure and an expected exteriorsurface of the first structure and/or (ii) between the first exteriorfeatures and expected exterior features of the first structure. One ormore parameters of the first machine learning model may be adjustedbased on the one or more differences.

In a ninth aspect according to any of the sixth through eighth aspects,training the first machine learning model to predict the interior modelsincludes receiving an exterior surface and exterior features of a firststructure from the plurality of structures and predicting, with thefirst machine learning model, a first interior model of the firststructure based on the exterior contour of the first structure. One ormore differences may be detected between the first interior model of thefirst structure and an expected interior model of the first structure.One or more parameters of the first machine learning model may beadjusted based on the one or more differences.

In a tenth aspect according to any of the first through ninth aspect,the structure includes at least one of: a building, a vehicle, aninfrastructure component, a ship, a spacecraft, an aircraft, a tank,and/or an appliance.

In an eleventh aspect, a method is provided that includes receivingtraining data for a plurality of structures. The training data mayinclude exterior imagery of the plurality of structures, expectedexterior surfaces for the plurality of structures, expected exteriorfeatures for the plurality of structures, and expected interior modelsfor the plurality of structures. A first machine learning model may betrained to generate predicted exterior surfaces and predicted exteriorfeatures for at least a subset of the plurality of structures based atleast on (i) exterior imagery of the subset of the plurality ofstructures and (ii) expected exterior surfaces of the subset of theplurality of structures. The first machine learning model may also betrained to predict interior models for at least the subset of theplurality of structures based at least on the predicted exteriorsurfaces and the predicted exterior features. The first machine learningmodel may be deployed to predict exterior contours and interior modelsfor additional structures separate from the plurality of structures.

In a twelfth aspect according to the eleventh aspect, the method furtherincludes, prior to receiving the training data, receiving a plurality ofarchitectural plans for the plurality of structures and generating, witha second machine learning model, the expected interior models for theplurality of structures based on the plurality of architectural plans.

In a thirteenth aspect according to any of the eleventh and twelfthaspects, training the first machine learning model to predict exteriorcontours and exterior features includes receiving first exterior imageryof a first structure from the plurality of structures and predicting,with the first machine learning model, a first exterior surface of thefirst structure and first exterior features based on the first exteriorimagery. One or more differences may be determined (i) between the firstexterior surface of the first structure and an expected exterior surfaceof the first structure and/or (ii) between the first exterior featuresand expected exterior features of the first structure. One or moreparameters of the first machine learning model may be adjusted based onthe one or more differences.

In a fourteenth aspect according to any of the eleventh throughthirteenth aspects, training the first machine learning model to predictthe interior models includes receiving an exterior surface and exteriorfeatures of a first structure from the plurality of structures andpredicting, with the first machine learning model, a first interiormodel of the first structure based on the exterior surface of the firststructure. One or more differences may be determined between the firstinterior model of the first structure and an expected interior model ofthe first structure. One or more parameters of the first machinelearning model may be adjusted based on the one or more differences.

In a fifteenth aspect according to the fourteenth aspect, the exteriorsurface of the first structure is one of an expected exterior surface ofthe first structure included within the training data and/or a predictedexterior surface of the first structure generated by the first machinelearning model. The exterior features of the first structure may be oneof expected exterior features of the first structure included within thetraining data and/or predicted exterior features of the first structuregenerated by the first machine learning model.

In a sixteenth aspect according to any of the eleventh through fifteenthaspects, training the first machine learning model to predict theinterior models includes training the first machine learning model togenerate interior models that comply with at least one of (i) spatialconstraints of the exterior surfaces and/or (ii) common constructionmethods and structural design requirements.

In a seventeenth aspect according to the sixteenth aspect, the at leastone of (i) the spatial constraints of the exterior surfaces and/or (ii)the common construction methods and structural design requirements arerepresented within an objective function for the first machine learningmodel.

In an eighteenth aspect according to any of the eleventh throughseventeenth aspects, deploying the first machine learning model furtherincludes receiving exterior imagery of a structure and determining, witha machine learning model, an exterior surface, exterior features, and aninterior model of the structure. A three-dimensional representation ofinterior portions of the structure and exterior portions of thestructure may be generated based on the exterior surface and theinterior model.

In a nineteenth aspect according to the eighteenth aspect, determiningthe exterior contours, exterior features, and an interior model of thestructure further includes determining, with a machine learning model,an exterior surface of the structure that encloses exterior portions ofthe structure depicted within the exterior imagery and determining, withthe machine learning model, exterior features of the structure based onthe exterior imaging and/or the exterior surface. An interior model ofthe structure may also be determined based on the exterior contours andthe exterior features using the machine learning model.

In a twentieth aspect according to any of the thirteenth throughnineteenth aspects, the exterior features include at least one of doors,windows, structural support elements, corners, and/or utility systems ofthe structure.

The features and advantages described herein are not all-inclusive and,in particular, many additional features and advantages will be apparentto one of ordinary skill in the art in view of the figures anddescription. Moreover, it should be noted that the language used in thespecification has been principally selected for readability andinstructional purposes, and not to limit the scope of the disclosedsubject matter.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates a system for predicting interior models of structuresaccording to an exemplary embodiment of the present disclosure.

FIG. 2 illustrates a machine learning model processing flow according toan exemplary embodiment of the present disclosure.

FIG. 3 illustrates features according to an exemplary embodiment of thepresent disclosure.

FIG. 4 illustrates an architectural plan according to an exemplaryembodiment of the present disclosure.

FIG. 5 illustrates a system for generating interior model training dataaccording to an exemplary embodiment of the present disclosure.

FIG. 6 illustrates a method for predicting interior models of structuresaccording to an exemplary embodiment of the present disclosure.

FIG. 7 illustrates a method for training a machine learning model topredict interior models according to an exemplary embodiment of thepresent disclosure.

FIG. 8 illustrates a computer system according to an exemplaryembodiment of the present disclosure.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

For certain buildings, it may be possible to look up accessarchitectural plans (e.g., blueprints, floor plans) that depict orotherwise represent interior layouts for the building. In suchscenarios, it may be possible to reconstruct an interior layout of thebuilding based on these plans. For example, U.S. patent application Ser.No. 17/487,838 describes various techniques for extracting andconstructing an interior layout of a building based on architecturalplans for the building.

However, in certain scenarios, it may not be feasible to locate copiesof architectural plans for certain structures. For example, in combatscenarios, individuals may not have access to architectural plans forbuildings prior to entry. In emergency response scenarios, there may notbe time to locate copies of the architectural plans for a building. Itmay further be appreciated that architectural plans for variousstructures may not be readily accessible for various other reasons(e.g., plans that have not been digitized, plans that are stored inprivate or siloed databases, plans that have been lost). Accordingly, itmay be necessary to rapidly reconstruct a structure's interior layoutusing other means.

In many situations, exterior imagery of a structure may be more readilyavailable. For example, overhead imagery (e.g., captured by a satelliteor UAV) and/or other imagery (e.g., captured by individuals located inview of the building) may be used to assist in determining a plausibleinterior layout for the structure. Architectural technicians may thenanalyze the exterior image using specialized software to makerepresentative models of structures based on standard constructiontemplates and common practices. These representative models (interiorand exterior models) may then be used for various types of structuralanalysis. However, these techniques are slow and cumbersome and oftencannot be done quickly enough for use in emergency scenarios. Therefore,there exists a need to expedite and automate this process such thatexterior imagery can be used to predict interior layouts in emergencyscenarios.

One solution to this problem is to analyze the exterior imagery usingone or more machine learning models. In one implementation, the machinelearning model may analyze the exterior imagery to determine an exteriorsurface of a structure depicted in the exterior imagery. The exteriorimagery and/or the exterior surface may then be used by the model toidentify one or more exterior features of the structure. In certaininstances, these exterior features may be common to both the interiorand exterior of the structure. The exterior surface and the exteriorfeatures may then be used to predict an interior model of the structurethat contains one or more interior features. These interior features mayinclude structural features (e.g., support beams, wall assemblies)and/or functional features for occupants of the structure (e.g., doors,windows, plumbing, HVAC components). The machine learning model may beconfigured to arrange the interior features for the structure such thatthe interior features align with any associated exterior features andfit within the exterior surface of the structure. In certain scenarios,the machine learning model may be trained based on the outputs of othermachine learning models. For example, a second machine learning modelmay be used to generate a training dataset of expected interior layoutsfor structures based on architectural plans for the structures. In suchinstances, exterior imagery of a plurality of structures may be combinedwith known interior layouts for those structures to form a uniquetraining dataset for the machine learning model.

FIG. 1 illustrates a system 100 for predicting interior models ofstructures according to an exemplary embodiment of the presentdisclosure. The system 100 may be configured to predict interior modelsfor interior portions of one or more structures 108. In particular, thesystem 100 may be configured to receive exterior imagery 112 of astructure 108 and generate a three-dimensional representation 122 of thestructure 108. The three-dimensional representation 122 may include athree-dimensional model (e.g., computer model, CAD model, athree-dimensional GIS model) of the exterior and interior of thestructure 108.

Although depicted as a building in FIG. 1 , in practice, the structure108 may include any type of structure (e.g., man-made structure). Forexample, the structure 108 may include any structure with an interiorspace that is at least partially enclosed. As a specific example, thestructure may include single-story buildings, multi-story building,warehouse structures, infrastructure facilities, outdoor structures(e.g., pavilions, gazebos, decks, bridges, dams), or combinationsthereof. As a further example, infrastructure facilities may includeinterior and exterior structures of dams, storm water pipes, sewerpipes, tunnels (e.g., access tunnels, tunnels for automobiles),channels, utility stations (e.g., pump stations), conduits (e.g.,electrical conduits), and the like. In still further implementations,the structures may include part or other components (e.g., mechanicalcomponents, chemical components, electrical components) of otherproducts or devices (e.g., vehicle components, aircraft components,artillery components, weapon components). Accordingly, any reference tobuildings herein should be understood to apply similarly to any type ofstructure. Similarly, the present disclosure uses the terms “blueprint”“architectural plan” and “plan” (and similar terms) to refer to plansfor buildings and other structures. One skilled in the art willunderstand that, in practice, these documents may be referred to usingdifferent terminology in other instances. For example, such documentsmay be referred to as “site plans,” “facility plans,” or other analogousterminology. As a further example, the plans discussed herein mayinclude one or more floor plans, elevation plans, circuit board layoutdiagrams, product design plans, and the like. As one specific example,the structure may include an engine of an aircraft, and the plan mayinclude a product design plan for the engine. As another specificexample, the structure may include an artillery weapon, and the plan mayinclude a multi-view structural plan or product design plan for theartillery weapon.

The exterior of the structure 108 may include any portion of thestructure 108 that is visible from outside of the structure 108 (e.g.,visible in exterior imagery of the structure 108). Additionally oralternatively, the exterior of the structure 108 may include the surfaceof the building that faces an exterior environment of the structure 108(e.g., an outdoor environment surrounding the structure 108). Theinterior of the structure 108 may include any portion of the structure108 that is not visible from outside of the structure 108 (e.g., visiblein exterior imagery of the structure 108). In certain instances,portions of the interior of the structure 108 may be visible in exteriorimagery (e.g., through windows of the structure 108). Additionally oralternatively, the interior of the structure 108 may include any portionof the structure 108 that is contained within the exterior of thestructure 108. In certain instances, the interior of the structure mayinclude portions of outer walls or support systems of the structure 108.For example, the exterior of the structure 108 may include an exteriorsurface of an outer concrete wall, and the interior of the structure 108may include interior portions of the concrete wall (e.g., not in directcontact with an outdoor area surrounding the structure 108) and othermaterials in the outer brick wall (e.g., support beams, internalsupporting materials for the concrete wall). In various implementations,the interior of the structure 208 may include one or more of structuralcomponents (a frame of the structure 108, load-bearing walls/beams/othersystem of the structure 108), interior/exterior construction materialsof the structure, material properties (e.g., strength undertension/compression, impact resistance, dimensions), structuraldetailing for the structure 108, and any other elements that contributeto the overall structural strength of the structure 108. In certainimplementations, the interior of the structure 108 may include elementsthat do not contribute to the structural strength of the structure 108,such as non-load-bearing walls, doors, windows, plumbing system, HVACsystems, fire suppression systems, and the like.

In particular, the system 100 may include a computing device 102configured to receive the exterior imagery 112 and generate thethree-dimensional representation 122. For example, computing device 102may receive the exterior imagery 112 from a database 104 storingexterior imagery 114, 116 for a plurality of different structures. Theexterior imagery 112, 114, 116 may represent one or more two- orthree-dimensional images depicting an exterior of the structure 108 and,in certain instances an outdoor area surrounding the structure 108. Theexterior imagery 112 may include any images of the exterior of thestructure 108. For example, the exterior imagery 112 may includeoverhead imagery (e.g., captured by satellite/UAV). As another example,the exterior imagery 112 may be captured from ground level (e.g., by anindividual within view of the structure 108). In certain instances, theexterior imagery 112 may be orthorectified such that the images can beused (e.g., by a user or computing process) as a basis for accuratespatial measurements of the structure. In further instances, theexterior imagery 112 may include three-dimensional imagery of theexterior of the structure 108.

The computing device 102 may contain a machine learning model 124configured to determine an exterior surface 126 and/or an interior model134 for the structure 108. For example, the machine learning model 124may determine, based on the exterior imagery, an exterior surface 126 ofthe structure 108. The exterior surface 126 may include athree-dimensional representation of the exterior of the structure 108.For example, the exterior surface 126 may include a three-dimensionalenvelope or contour approximating the shape of the exterior of thestructure 108. The machine learning model 124 may additionally identifyone or more exterior features 132 depicted within the exterior imagery112 and/or based on the exterior surface 126. In certainimplementations, the exterior features 132 may include one or morefunctional or aesthetic features of the structure 108 that areexteriorly visible on the structure 108. In certain implementations, atleast a subset of the exterior features 132 may be common to an exteriorof the structure and interior of the structure. For example, theexterior features 132 may include windows, doors, exterior vents, outerwall materials, visible/exposed support elements, and the like.

Based on the exterior surface 126 and the exterior models 132, themachine learning model 124 may determine an interior model 134. Theinterior model 134 may represent a predicted layout or structural planof an interior portion of the structure 108. For example, the interiormodel 134 may represent a two-dimensional and/or a three-dimensionalrepresentation of an interior portion of the structure 108. Inparticular, the interior model 134 may be generated to comply with oneor more spatial constraints of the exterior surface 126 and commonconstruction methods and structural design requirements (e.g.,constraints representative of well established, common constructionmethods, and standard construction templates and structural design rulesof thumb). For example, the interior model 134 may be generated bycombining a plurality of interior features that are common to structures(e.g., structures of the same type as the structure 108). For example,where the exterior features 132 indicate that the structure 108 has aconcrete exterior wall, the interior features 140 may include a commonconcrete wall assembly, selected based on a size (e.g., length andheight) of the wall. As another example, where the exterior features 132indicate that the structure 108 has glass exterior walls, the interiorfeatures 140 may include support structures (e.g., support frames)common to buildings with glass exterior walls. The interior layoutfeatures may then be combined and arranged to comply with one or morerestrictions indicated by the exterior surface 126 and/or the exteriorfeatures 132. In particular, the interior features 140 may be generatedto align with one or more exterior features 132. As a particularexample, the interior model 134 may be generated to contain rooms thatalign with certain exterior features 132, such as windows and exteriorair conditioning units. Additionally or alternatively, the interiorfeatures 140 may be generated to fit within the exterior surface 126.For example, the exterior surface 126 may define boundaries for exteriorwalls of the interior model 134. Accordingly, rooms within the interiormodel 134 (and the interior features 140 contained within) may begenerated to fit within the boundaries indicated by the exterior surface126. In certain implementations, to comply with the exterior surface 126and the exterior features 132, the machine learning model 124 maygenerate a plurality of interior features 140 based on commonconstruction practices. The machine learning model 124 may then combineand arrange these interior features 140 to fit within the exteriorsurface 126 and to align with one or more exterior features 132 that arecommon to both an exterior and an interior of the structure 108.

In certain implementations, the machine learning model 124 may determinethe exterior surface 126, exterior features 132, and interior model 134in a particular order based on the received exterior imagery 112. Forexample, FIG. 2 illustrates a machine learning model processing flow 200according to an exemplary embodiment of the present disclosure. In themachine learning model processing flow 200, the machine learning model124 receives the exterior imagery 112. Based on the exterior imagery112, the machine learning model 124 may identify an exterior surface 126for the structure 108. In particular, the machine learning model 124 mayconstruct a three-dimensional bounding surface for the structure 108depicted within the exterior imagery 112. For example, the machinelearning model 124 may align multiple images of the structure 108 basedon various visual/spatial features and extract, based on the alignedimages, an approximate exterior surface that encompasses the depictedportions of the structure 108. The machine learning model 124 may thenidentify exterior features 132 based on the exterior imagery 112 and/orthe exterior surface 126. For example, the machine learning model 124may include a neural network (e.g., convolutional neural network,recurrent neural network) configured to identify certain types ofpredetermined features depicted within the exterior imagery 112,informed by the associated contours of the features in the exteriorsurface 126. Then, based on the exterior surface 126 and the exteriorfeatures 132, the machine learning model 124 may determine the interiormodel 134 for the structure 108, using techniques discussed above.

The actual exterior features 132 and interior features 140 generated bythe machine and model 124 may contain one or more different types ofinformation. For example, FIG. 3 illustrates features 302, 304 accordingto an exemplary embodiment of the present disclosure. In particular,FIG. 3 depicts an interior feature 302, which may be an exemplaryimplementation of one of the interior features 140, and an exteriorfeature 304, which may be an exemplary implementation of one of theexterior features 132. Each of the features 302, 304 includes a label306, 308, physical dimensions 310, 312, materials 314, 316, and one ormore adjacent elements 318, 320. The label 306, 308 may indicate a titleor type of feature for the features 302, 304. For example, the label 306for the interior feature 302 may identify the feature 302 as an“interior door.” As another example, the label 308 for the exteriorfeature 304 may identify the feature 304 as a “window.” In additional oralternative implementations, the labels 306, 308 may provide furtherinformation regarding the features 302, 304. For example, the features302, 304 may include information regarding the material properties ofthe materials used to construct the features (e.g., strength undertension/compression, impact resistance, assembly techniques). In certainimplementations, the labels 306, 308 may include a unique identifier(e.g., unique alphanumeric identifier) of the features 302, 304, whichmay be used to uniquely refer to the specific feature 302, 304 elsewherewithin the interior model 134.

The physical dimensions 310, 312 may indicate one or more physicaldimensions (e.g., length, width, height, thickness) for the features302, 304. For example, the physical dimensions 312 may include a heightand width for a window (e.g., the exterior feature 304). As anotherexample, the physical dimensions 310 may include the height and width ofan interior door (e.g., the interior feature 302). In additional oralternative implementations, physical dimensions 310, 312 may includeone or more of a length and thickness of a support beam and/or a lengthand thickness of an interior wall assembly.

The materials 314, 316 may indicate one or more construction materialsused to form the features 302, 304. For example, where the features 302,304 correspond to individual structural elements (e.g., support beams,windows, doors, drywall, ducts), the materials 314, 316 may indicate thematerials from which the individual elements are made. Additionally oralternatively, the features 302, 304 may correspond to certainassemblies (e.g., wall assemblies). In such instances, the materials314, 316 indicate a type of assembly, which may specify one or morematerials used to construct the assembly (e.g., size and spacing ofsupport beams, use of brick, concrete, drywall, thickness of anycladding material used).

Additionally or alternatively, the features 302, 304 may specify one ormore adjacent elements 318, 320. For example, the adjacent elements 318,320 may identify interior features and/or exterior features that areimmediately adjacent to the features 302, 304. For example, where theinterior feature 302 is a door, the adjacent elements 318 may includeunique identifiers (e.g., labels 306) for walls that are adjacent to thedoor. As another example, where the exterior feature 304 is an exteriorHVAC unit, the adjacent elements 320 may identify adjacent structuralelements (e.g., adjacent walls and/or roofs) and/or adjacent HVACelements (e.g., ducts connected to the exterior HVAC unit).

It should be understood that the above examples of labels 306, 308,physical dimensions 310, 312, materials 314, 316, and adjacent elements318, 320 are merely illustrative. Accordingly, based on the presentdisclosure, one skilled in the art may recognize additional oralternative labels, physical dimensions, materials, and/or adjacentelements that may be used to describe interior features and/or exteriorfeatures. Furthermore, one skilled in the art may understand that thefeatures 302, 304 may contain additional or alternative information,including omitting one or more of the labels, physical dimensions,materials, and/or adjacent elements and including adding one or moreadditional fields to the features 302, 304. All such implementations arehereby considered within the scope of the present disclosure.

Returning to FIG. 1 , the machine learning model 124 may generate theinterior model 134 to contain interior features and/or exterior features132 as discussed above in connection with FIG. 3 . Furthermore, however,the machine learning model 124 may generate the interior model 134 toinclude a visual or structural depiction of the structure 108. Inparticular, the interior model 134 may contain a visual depiction of theinterior features 140 arranged to comply with the exterior surface 126and exterior features 132, as described above. In certainimplementations, the interior model 134 may be generated as atwo-dimensional representation of the structure 108.

For example, FIG. 4 illustrates portion of an architectural plan 400according to an exemplary embodiment of the present disclosure. The plan400 may be an exemplary implementation of all or part of atwo-dimensional interior model 134 that may be generated by a machinelearning model 124. The plan 400 includes various elements of a buildingstructure, a portion of which are identified using reference numeralsfor discussion below. The plan 400 as depicted may be a part of a floorof a building. The plan 400 includes depictions of exterior walls 404,414 and interior walls 402, 406, 410 (only a subset of which arenumbered for clarity). The interior walls 402, 406, 410 include twodifferent types of walls: interior partition walls 402, 406 and interiorload-bearing walls 410. The plan 400 also includes a depiction of afoundation structure 408, along with structural ties 412 connectingother parts of the building (e.g., the interior load-bearing wall 410)to the foundation structure 408. The exterior walls 404, 414,load-bearing walls 410, foundation structure 408, and structural ties412 may all be interior features 140 of a structure.

In certain implementations, not all of the depicted features may benecessary to properly determine an interior layout of the building. Forexample, the foundation structure 408 and the structural ties 412 maynot be necessary to accurately determine the interior layout of thefloor. Accordingly, in certain implementations, the machine learningmodel 134 may not be trained to include foundation structures 408 and/orstructural ties 412 as interior features 140 in an interior model 134.For clarity, the plan 400 includes bounding boxes 418, 420, 422, 424around corresponding elements 202, 204, 208, 216, but such boundingboxes may not be included within the interior model 134 of acorresponding structure 108.

Returning to FIG. 1 , the computing device 102 may then generate athree-dimensional representation 122 of the structure 108 based on theinterior model 134. For example, where the interior model 134 is atwo-dimensional representation of an interior of the structure 108, thecomputing device 102 may extrude individual interior features 140 withinthe interior model 134 to generate the three-dimensional representation122. In particular, the interior features 140 may be extruded accordingto physical dimensions 310 stored in association with the interiorfeatures 140. For example, the physical dimensions 310 may specify aheight for one or more individual interior features 140. In suchinstances, the computing device 102 may extrude individual interiorfeatures 140 according to the specified heights while generating thethree-dimensional representation 122. Furthermore, the computing device102 may join the adjacent interior features 140 within thethree-dimensional representation 122 (e.g., by extruding betweenadjacent elements). In particular, adjacent elements may be identifiedbased on corresponding information stored within the interior features140 (e.g., adjacent elements 318, 320).

In certain instances, the structure 108 may have more than one floor orlevel. For example, the computing device 102 may determine that thestructure 108 includes multiple levels based on exterior dimensions ofthe structure 108 determined based on the exterior imagery 112 (e.g., aheight greater than a certain predetermined threshold, such as 15 feet,30 feet, 45 feet). Additionally or alternatively, the computing device102 may determine that the structure 108 includes multiple levels basedon exterior features 132 identified within the exterior imagery 112. Forexample, the computing device 102 may detect horizontal arrangements ofwindows at a plurality of heights (e.g., three different heights). Insuch instances, the computing device 102 may determine that thestructure 108 has multiple levels (e.g., three different levels).

For a structure 108 that has multiple levels, the machine learning model124 may be configured to generate a plurality of interior models 134.For example, the machine learning model 124 may generate separateinterior models 134 for each of the separate levels detected within thestructure 108. In such instances, when generating the three-dimensionalrepresentation 122 of the structure 108, the computing device 102 maycombine a plurality of three-dimensional representations of individualfloors or levels of the structure 108. For example, thethree-dimensional representation 122 may be formed by combining threeseparate three-dimensional representations of each of the three floors,according to the order of the floors within the structure (e.g., fromlowest to highest).

In certain implementations, the three-dimensional representation maycontain structural information regarding the structure 108. For example,the three-dimensional representation 122 may store materials 314, 316for individual elements (e.g., individual interior features, individualexterior features) within the three-dimensional representation 122. Inparticular, each individual element or feature within thethree-dimensional representation 122 may contain the correspondinginformation (e.g., labels, physical dimensions, materials, adjacentelements) stored in association with the corresponding interior feature140 that was extruded.

Returning to the machine learning model 124, FIG. 1 depicts the machinelearning model 124 as a single machine learning model. However, incertain implementations, the machine learning model 124 may be depictedas more than one model. For example, in certain implementations, themachine learning model 124 may be implemented as three separate machinelearning models: a first model to generate the exterior surface 126, asecond model to identify the exterior features 132, and a third model togenerate the interior model 134. Furthermore, it should be understoodthat, although the terms “machine learning model” and “interior model”both contain the word “model,” the interior model 134 should not beunderstood to include machine learning or other predictive models. Inparticular, as explained above, the interior model 134 should beunderstood to contain a representation (e.g., a two-dimensional and/or athree-dimensional representation) of the interior of the structure 108.

Furthermore, the machine learning model 124 may be trained in order toaccurately generate an exterior surface 126, exterior features 132, andan interior model 134. In particular, the machine learning model 124 maybe trained by the computing device 102 and/or another computing devicebased on data contained within a training database 106. In particular,the training database 106 may contain exterior imagery 118, 120 storedin association with expected exterior surfaces 128, 130, expectedexterior features 146, 148, an expected interior models 136, 138.Techniques for training the machine learning model 124 using data from atraining database 106 are discussed in greater detail below inconnection with the method 700 and FIG. 7 .

In certain implementations, multiple structures may be depicted withinthe exterior imagery 112. In such instances, the computing device 102may detect each of the multiple structures 108 and may repeat theprocessing for each of the structures 108 identified within the imagery.For example, the machine learning model 124 may detect multiple exteriorsurfaces 126 for multiple structures 108 within the exterior imagery112. Upon detecting the multiple structures, the machine learning model124 may then proceed with identifying exterior surfaces 126 and exteriorfeatures 132 for each of the multiple structures. The machine learningmodel 124 may also proceed with generating interior model 134 for eachof the individual structures. This processing may be similar to thetechniques discussed above in connection with the structure 108. Incertain instances, the multiple structures may be processed one at atime (e.g., in an order in which they are detected within the exteriorimagery 112). Additionally or alternatively, the structures may beprocessed at least partially in parallel. For example, the computingdevice 102 may execute multiple instances of the machine learning model124, where each instance is responsible for processing a singlestructure detected within the exterior imagery 112. As another example,the machine learning model 124 may identify exterior surfaces 126 foreach of the identified structures before identifying exterior features132 for each of the multiple structures, and then may finally generateinterior models 134 for each of the multiple structures.

The computing device 102 also includes a processor 142 and a memory 144.The processor 142 and the memory 144 may implement one or more aspectsof the computing device 102. For example, the memory 144 may store oneor more instructions which, when executed by the processor 142, maycause the processor 142 to perform one or more operational features ofthe computing device 102 (e.g., implement the machine learning model518). The processor 142 may be implemented as one or more centralprocessing units (CPUs), field programmable gate arrays (FPGAs), and/orgraphics processing units (GPUs) configured to execute instructionsstored on the memory 144. Additionally, the computing device 102 may beconfigured to communicate (e.g., with the database 104 and/or thetraining database 106) using a network 508. For example, the computingdevice 102 may communicate with the network 508 using one or more wirednetwork interfaces (e.g., Ethernet interfaces) and/or wireless networkinterfaces (e.g., Wi-Fi®, Bluetooth®, and/or cellular data interfaces).In certain instances, the network may be implemented as a local network(e.g., a local area network), a virtual private network, L1, and/or aglobal network (e.g., the Internet).

FIG. 5 illustrates a system 500 for generating interior model trainingdata according to an exemplary embodiment of the present disclosure. Forexample, the system 500 may be configured to generate at least a portionof the training data stored within a training database 106 of the system100. In particular, the system 500 may be configured to generate theexpected interior models 136, 138 discussed above. The system 500includes a computing device 502, a database 504, and a training database506. The training database 506 may be an exemplary implementation of thetraining database 106. The training database 506 stories interior models524, 526, which may be exemplary implementations of the expectedinterior models 136, 138.

The computing device 502 may receive architectural plans 510 and maygenerate one or more interior models 522 based on the architecturalplans 510. In particular, the architectural plans 510 may include one ormore floor plans 516 for a structure, and the computing device 502 maygenerate interior models 522 for the structure based on the floor plans516. The computing device 502 may receive the architectural plans 510from a database 504 configured to store architectural plans 512, 514.For example, the database 504 may be stored blueprints, constructionplans, and/or any other architectural plans concerning the multiplestructures. In particular, in certain instances, the database 504 may bea governmental or commercial database of architectural plans 512, 514.

The computing device 502 may use a machine learning model 518 togenerate the interior model 522. In preferred embodiments, the machinelearning model 518 is separate from the machine learning model 124. Themachine learning model 518 may detect one or more interior featureswithin the floor plans 516 and may generate interior models 522 based onthe interior features and the relative positions within the floor plans516. In particular, the machine learning model 518 may be configured toidentify which sheets within the architectural plans 510 contain ordepict floor plans 516 and associated structure. The machine learningmodel 518 may then use one or more image processing techniques to detectand extract the individual features within the floor plans 516.Detecting and extracting the interior features 520 may includeidentifying various types of information regarding the interiorfeatures, including labels, physical dimensions, materials, adjacentelements, similar to the interior features 140, 302, 304 discussedabove. The machine learning model 518 may then construct an interiormodel 522 that contains the interior features 520 based on the relativepositions of the interior features 520 within the floor plans 516.Additional details regarding the techniques used to detect and generatethe interior features 520 and interior model 522 are further describedin U.S. patent application Ser. No. 17/487,838, filed on Sep. 28, 2021,and entitled “Generating Vector Versions of Structural Plans,” theentirety of which is incorporated by reference herein for all purposes.

Once the interior model 522 is generated, the interior model 522 maythen be used to train other machine learning models. For example, theinterior model 522 may be stored in a training database 506 thatcontains a plurality of interior models 524, 526. These interior models522, 524, 526 may be used to train machine learning models for variouspurposes. In one particular instance, the interior models 522, 524, 526may be stored within the training database 506 in association withexterior imagery of the corresponding structures. In such instances, theinterior models 522, 524, 526 may be used to train the machine learningmodel 124. In particular, the interior models 522, 524, 526 may beexemplary implementations of the expected interior models 136, 138 inthe training database 106.

The computing device 502 also includes a processor 528 and a memory 530.The processor 528 and the memory 530 may implement one or more aspectsof the computing device 502. For example, the memory 530 may store oneor more instructions which, when executed by the processor 528, maycause the processor 528 to perform one or more operational features ofthe computing device 502 (e.g., implement the machine learning model518). The processor 528 may be implemented as one or more centralprocessing units (CPUs), field programmable gate arrays (FPGAs), and/orgraphics processing units (GPUs) configured to execute instructionsstored on the memory 530. Additionally, the computing device 502 may beconfigured to communicate (e.g., with the database 504 and/or thetraining database 506) using a network 508. For example, the computingdevice 502 may communicate with the network 508 using one or more wirednetwork interfaces (e.g., Ethernet interfaces) and/or wireless networkinterfaces (e.g., Wi-Fi®, Bluetooth®, and/or cellular data interfaces).In certain instances, the network may be implemented as a local network(e.g., a local area network), a virtual private network, L1, and/or aglobal network (e.g., the Internet).

In certain instances, all or part of the systems 100, 500 may becombined. For example, as explained previously, the training database506 may be at least partially implemented by the training database 106.Additionally or alternatively, the computing devices 102, 502 may beimplemented by the same computing device. In still furtherimplementations, the systems 100, 500 may be implemented in adistributed computing environment (e.g., a cloud computing environment).In such instances, each of the systems 100, 500 may be implemented byone or more (e.g., a plurality of) computing devices within thedistributed computing environment (e.g., within one or more clusters ofthe distributed computing environment).

FIG. 6 illustrates a method 600 for predicting interior models ofstructures according to an exemplary embodiment of the presentdisclosure. In particular, the method 600 may be performed to predictinterior models 134 for structures 108 based on exterior imagery 112 ofthe structures. The method 600 may be implemented on a computer system,such as the system 100. For example, the method 600 may be implementedby the computing device 102. The method 600 may also be implemented by aset of instructions stored on a computer readable medium that, whenexecuted by a processor, cause the computer system to perform the method600. For example, all or part of the method 600 may be implemented bythe processor 142 and the memory 144. Although the examples below aredescribed with reference to the flowchart illustrated in FIG. 6 , manyother methods of performing the acts associated with FIG. 6 may be used.For example, the order of some of the blocks may be changed, certainblocks may be combined with other blocks, one or more of the blocks maybe repeated, and some of the blocks described may be optional.

The method 600 may begin with receiving exterior imagery of a structure(block 602). For example, the computing device 102 may receive exteriorimagery 112 of a structure 108. The exterior imagery 112 may depict oneor more exterior surfaces of the structure 108. For example, theexterior imagery 112 may depict an exterior surface of the structure 108as visible from above, below, or outside of the structure 108. Incertain implementations, as explained above, the exterior imagery 112may depict a three-dimensional exterior view of the structure 108.

An exterior surface of the structure may be determined (block 604). Forexample, the heating device 102 may determine an exterior surface 126 ofthe structure 108. In certain implementations, the exterior surface 126may include a three-dimensional representation of the exteriordimensions of the structure 108. For example, the exterior surface 126may represent exterior contours of the building as visible from withinthe exterior imagery 112. Accordingly, the exterior surface 126 mayinclude contours or other representations of various exterior features132 on the structure 108 (e.g., decorative features, functionalfeatures, structural features). The exterior surface 126 may bedetermined using a machine learning model 124. For example, the machinelearning model 124 may be configured to extract the exterior surface 126from one or more exterior images of the structure 108 contained withinthe exterior imagery 112. In implementations where the exterior imagery112 includes a three-dimensional representation (e.g., athree-dimensional image) of the structure 108, determining the exteriorsurface 126 may include extracting the three-dimensional contour of thestructure 108 from the contours of other structures in the surroundingarea within the exterior imagery 112. In implementations where theexterior imagery 112 does not include a three-dimensional representationof the structure 108, determining the exterior surface 126 may includecombining multiple views of the structure 108 into the exterior surface126.

Exterior features of the structure may be determined (block 606). Forexample, computing device 102 may determine exterior features 132 of thestructure 108. The exterior features 132 may be identified from withinthe exterior imagery 112. For example, the computing device 102 may usea machine learning model 124 to identify exterior features 132. In oneparticular instance, the machine learning model 124 may be configured toidentify doors, windows, visible structural beams, HVAC systemcomponents, and exterior plumbing fixtures as exterior features 132. Theexterior features 132 may be based on visual depictions of the exteriorof the structure 108 within the exterior imagery 112. Furthermore, theexterior features 132 may be identified within the exterior surface 126and/or a three-dimensional representation of the structure 108 withinthe exterior imagery 112 (where included).

An interior model of the structure may be determined based on theexterior surface and the exterior features (block 608). For example, thecomputing device 102 may determine an interior model 134 of thestructure 108 based on the exterior surface 126 and the exteriorfeatures 132. For example, as explained above, the interior model 134may be generated by predicting one or more interior features 140contained within the structure 108. The interior features 140 may bepredicted according to one or more of the exterior features 132, commonconstruction practices for structures similar to the structure 108,regulatory requirements for structures similar to the structure 108,structural design rules of thumb, and the like. The interior features140 may then be arranged within the interior model 134 based on theexterior surface 126 and/or the exterior features 132. For example,where the exterior features 132 include one or more features that arecommon to both the interior and exterior of the building (e.g., doors,windows, HVAC ducts), the interior features 140 may be arranged to alignwith corresponding exterior features 132 (e.g., so that interior doorsand windows on exterior walls align with exterior doors and windows). Asanother example, the interior features 140 may be arranged to fit withindimensions specified by the exterior surface 126 (e.g., to fit withinthe interior space of the structure 108, as indicated by the exteriordimensions of the structure 108 within the exterior surface 126. Theinterior model 134 and then be generated as a representation (e.g., athree-dimensional representation, a two-dimensional representation) ofthe interior features 140.

A three-dimensional representation of the structure may be generated(block 610). For example, the computing device 102 may generate athree-dimensional representation 122 of the structure 108 based on theexterior structure 126 and/or the interior model 134. In particular, asexplained above, the interior features 140 and/or exterior features 132may specify physical dimensions 310, 312 for individual features 140,132. The computing device 102 may accordingly be configured to extrudethe exterior features 132 and the interior features 140 according tothese physical dimensions 310, 312. In additional or alternativeimplementations, the interior model 134 may be generated as athree-dimensional representation of the interior of the structure 108.In such instances, the computing device 102 may generate athree-dimensional representation 122 of the structure 108 by combiningthe three-dimensional exterior surface 126 of the structure 108 with thethree-dimensional interior model 134 of the structure 108.

The method 600 accordingly enables computing devices to predict interiormodels for buildings and other structures without having to see insideof structures or analyze interior plans for the structures. The interiormodels generated according to the method 600 may be used for structuralor other analysis of the structures. For example, predicted interiorlayouts represented by the interior models may be used by emergencyresponse teams (e.g., paramedics, firefighters, police officers) topredict an interior layout of the structure in which an emergency istaking place. Such plans may then be used by the emergency responseteams in navigating the interior of the structure to locate and resolvethe emergency. Interior models may also be useful in combat scenarios.For example, military operatives may utilize the interior models toassist in navigating building interiors, similar to the emergencyresponse teams above. As another example, the interior models may beused in a destructive analysis of the structure (e.g., predicting theminimum amount of munitions necessary to destroy or incapacitate thestructure 108, to reduce collateral damage from the structure 108falling on other, nearby structures). As explained above, previoussystems for performing these functions typically relied on interiorviews or interior plans of the structures, or relied on the manualefforts of specialized technicians, which may be unavailable or too slowfor use in emergency or combat settings.

FIG. 7 illustrates a method 700 for training a machine learning model topredict interior models according to an exemplary embodiment of thepresent disclosure. In particular, the method 700 may be used to train amachine learning model, such as the machine learning model 124 topredict interior models 134 for structures 108 based on exterior imagery112 of the structures 108. The method 700 may be implemented on acomputer system, such as the system 500. For example, the method 700 maybe implemented by the computing device 502. The method 700 may also beimplemented by a set of instructions stored on a computer readablemedium that, when executed by a processor, cause the computer system toperform the method 700. For example, all or part of the method 700 maybe implemented by the processor 142 and the memory 144. Although theexamples below are described with reference to the flowchart illustratedin FIG. 7 , many other methods of performing the acts associated withFIG. 7 may be used. For example, the order of some of the blocks may bechanged, certain blocks may be combined with other blocks, one or moreof the blocks may be repeated, and some of the blocks described may beoptional.

The method 700 may begin with receiving training data for a plurality ofstructures (block 702). For example, computing device 102 (or anothercomputing device) may receive training data for a plurality ofstructures. The training data may be stored within a training database106. For example, the training data may include exterior imagery 118,120 of a plurality of structures. The training data may further includeexpected interior models 136, 138 and expected exterior surfaces 128,130 of the structures, along with expected exterior features 146, 148 ofthe structures. In certain implementations, the expected interior models136, 138 may be generated by another machine learning model 518. Forexample, as discussed in greater detail above in connection with thesystem 500, the expected interior models may be generated by anothermachine learning model 518 based on architectural plans.

A first machine learning model may be trained to generate predictedexterior surfaces and predicted exterior features (block 704). Forexample, a first machine learning model 124 may be trained to predictexterior surfaces 126 and exterior features 132 for a structure 108based on exterior imagery 112 of the structure 108. Training the machinelearning model 124 in this way may include providing exterior imagery118 to the machine learning model 124 and having the machine learningmodel 124 generate one or more predicted exterior surfaces in exteriorfeatures based on the exterior imagery 118, 120 the predicted exteriorsurfaces generated by the machine learning model 124 may be compared toexpected exterior surfaces 128, 130 associated with the exterior imagery118, 120 within the training data. Similarly, the predicted exteriorfeatures may be compared to the expected exterior features 146, 148associated with the exterior imagery 118, 120 within the training data.One or more differences may be detected between the predicted exteriorsurfaces and the expected exterior surfaces 128, 130 and/or between thepredicted exterior features and the expected exterior features 146, 148.For example, the predicted exterior surface may includethree-dimensional geometry that differs from the geometry in acorresponding expected exterior surface. As another example, thepredicted exterior features may include a feature not present incorresponding expected exterior features, or may not include a featurethat is present in the expected exterior features. As a further example,one or more aspects (e.g., metadata as in FIG. 3 ) of a predictedexterior feature may differ from aspects of a corresponding expectedexterior feature. Based on these differences, one or more parameters ofthe machine learning model 124 may be adjusted. For example, one or moreweights associated with individual features (e.g., individual spatialfeatures within the exterior imagery) may be adjusted. Additionally oralternatively, one or more individual features may be added or removedto the machine learning model 124.

The first machine learning model may also be trained to predict interiormodels (block 706). For example, the machine learning model 124 may betrained to predict interior models 134 of structures. Training themachine learning model 124 in this manner may include comparing interiormodels generated by the machine learning model to expected interiormodels 136, 138. For example, the computing device 102 (or anothercomputing device) may provide exterior imagery 118, 120 to the machinelearning model 124. The machine learning model 124 may generatepredicted interior models for the corresponding structures based on theexterior imagery 118, 120. For example, the machine learning model 124may generate a predicted exterior surface and a predicted exteriorfeature for each set of exterior imagery 118, 120 that is received.Based on the imagery 118, 120, the predicted exterior surfaces, and/orthe predicted exterior features, the machine learning model 124 may thengenerate predicted interior models for the corresponding structures.These predicted interior models may then be compared to the expectedinterior models 136, 138 corresponding to each exterior imagery 118,120. One or more differences may then be identified between thepredicted interior models and the expected interior models 136, 138. Forexample, the predicted interior model may include an interior featurenot present in a corresponding expected interior model, or may notinclude an interior feature that is included in the correspondingexpected interior model. As another example, one or more aspects (e.g.,metadata as in FIG. 3 ) of a predicted interior feature may differ fromaspects of a corresponding expected interior feature. As a furtherexample, one or more interior features may be in a different locationwithin the predicted interior model than in a corresponding expectedinterior model. Based on these differences, one or more parameters ofthe machine learning model 124 may be adjusted. For example, one or moreweights associated with individual features (e.g., individual spatialfeatures within the exterior imagery) may be adjusted. Additionally oralternatively, one or more individual features may be added or removedto the machine learning model 124. In certain instances, the machinelearning model 124 may be trained according to one or more objectivefunctions (e.g., to maximize the objective functions). In certaininstances, these objective functions may be formulated to enforcecertain constraints (e.g., to ensure that all interior features fitwithin a corresponding portion of an exterior surface). Otherconstraints may include ensuring that interior features align withcorresponding exterior features, where appropriate, or that anarrangement of certain interior features (e.g., structural assemblies)comply with common construction practices, requirements, and/or rules ofthumb.

The first machine learning model may deploy the machine learning modelto predict exterior surfaces and interior models (block 708). Forexample, the machine learning model 124 may be deployed to predictexterior surfaces 126 and interior models 134. For example, the machinelearning model 124 may be deployed within the computing device 102 foruse in predicting exterior surfaces and interior models in real timebased on exterior imagery of structures separate from those used intraining the machine learning model 124. For example, a user may captureor otherwise identify (e.g., within the database 104) exterior imagery112 for a structure 108. The exterior imagery 112 may then be providedto the computing device 102, which may then utilize the machine learningmodel 124 to predict an exterior surface 126 and interior model 134 forthe structure 108. In certain instances, the exterior surface 126 andthe interior model 134 may then be used to generate a three-dimensionalrepresentation 122 of the structure 108, as discussed above.

The method 700 thus enables computing devices to train machine learningmodels for use in predicting exterior surfaces and interior models ofstructures. As explained further above, such representations ofstructures may be useful in many scenarios (e.g., combat scenarios,emergency response scenarios). Furthermore, in certain implementations,the method 700 relies on unique training data received from a secondmachine learning model that is capable of generating interior models ofbuildings based solely on architectural plans for those buildings. Sucha system dramatically increases the available training data for themachine learning model 124 trained in the method 700. Accordingly, themachine learning models trained in this way may be significantly moreaccurate in their interior model predictions than models relying ontraditionally available training data.

Furthermore, the method 700 is flexible enough to be used with differenttypes of model architectures. For example, in certain implementations,the machine learning model 124 may be implemented as more than oneindividual machine learning model. For example, a first machine learningmodel may be used to predict exterior surfaces 126 and exterior features132 and a second machine learning model may be used to predict interiormodels 134. In such instances, the block 704 may be performed to trainthe first machine learning model, and the block 706 may be performed totrain the second machine learning model.

FIG. 8 illustrates an example computer system 800 that may be utilizedto implement one or more of the devices and/or components discussedherein, such as the computing devices 102, 502, databases 104, 504, andtraining databases 106, 506. In particular embodiments, one or morecomputer systems 800 perform one or more steps of one or more methodsdescribed or illustrated herein. In particular embodiments, one or morecomputer systems 800 provide the functionalities described orillustrated herein. In particular embodiments, software running on oneor more computer systems 800 performs one or more steps of one or moremethods described or illustrated herein or provides the functionalitiesdescribed or illustrated herein. Particular embodiments include one ormore portions of one or more computer systems 800. Herein, a referenceto a computer system may encompass a computing device, and vice versa,where appropriate. Moreover, a reference to a computer system mayencompass one or more computer systems, where appropriate.

This disclosure contemplates any suitable number of computer systems800. This disclosure contemplates the computer system 800 taking anysuitable physical form. As example and not by way of limitation, thecomputer system 800 may be an embedded computer system, a system-on-chip(SOC), a single-board computer system (SBC) (such as, for example, acomputer-on-module (COM) or system-on-module (SOM)), a desktop computersystem, a laptop or notebook computer system, an interactive kiosk, amainframe, a mesh of computer systems, a mobile telephone, a personaldigital assistant (PDA), a server, a tablet computer system, anaugmented/virtual reality device, or a combination of two or more ofthese. Where appropriate, the computer system 800 may include one ormore computer systems 800; be unitary or distributed; span multiplelocations; span multiple machines; span multiple data centers; or residein a cloud, which may include one or more cloud components in one ormore networks. Where appropriate, one or more computer systems 800 mayperform without substantial spatial or temporal limitation one or moresteps of one or more methods described or illustrated herein. As anexample and not by way of limitation, one or more computer systems 800may perform in real time or in batch mode one or more steps of one ormore methods described or illustrated herein. One or more computersystems 800 may perform at different times or at different locations oneor more steps of one or more methods described or illustrated herein,where appropriate.

In particular embodiments, computer system 800 includes a processor 806,memory 804, storage 808, an input/output (I/O) interface 810, and acommunication interface 812. Although this disclosure describes andillustrates a particular computer system having a particular number ofparticular components in a particular arrangement, this disclosurecontemplates any suitable computer system having any suitable number ofany suitable components in any suitable arrangement.

In particular embodiments, the processor 806 includes hardware forexecuting instructions, such as those making up a computer program. Asan example and not by way of limitation, to execute instructions, theprocessor 806 may retrieve (or fetch) the instructions from an internalregister, an internal cache, memory 804, or storage 808; decode andexecute the instructions; and then write one or more results to aninternal register, internal cache, memory 804, or storage 808. Inparticular embodiments, the processor 806 may include one or moreinternal caches for data, instructions, or addresses. This disclosurecontemplates the processor 806 including any suitable number of anysuitable internal caches, where appropriate. As an example and not byway of limitation, the processor 806 may include one or more instructioncaches, one or more data caches, and one or more translation lookasidebuffers (TLBs). Instructions in the instruction caches may be copies ofinstructions in memory 804 or storage 808, and the instruction cachesmay speed up retrieval of those instructions by the processor 806. Datain the data caches may be copies of data in memory 804 or storage 808that are to be operated on by computer instructions; the results ofprevious instructions executed by the processor 806 that are accessibleto subsequent instructions or for writing to memory 804 or storage 808;or any other suitable data. The data caches may speed up read or writeoperations by the processor 806. The TLBs may speed up virtual-addresstranslation for the processor 806. In particular embodiments, processor806 may include one or more internal registers for data, instructions,or addresses. This disclosure contemplates the processor 806 includingany suitable number of any suitable internal registers, whereappropriate. Where appropriate, the processor 806 may include one ormore arithmetic logic units (ALUs), be a multi-core processor, orinclude one or more processors 806. Although this disclosure describesand illustrates a particular processor, this disclosure contemplates anysuitable processor.

In particular embodiments, the memory 804 includes main memory forstoring instructions for the processor 806 to execute or data forprocessor 806 to operate on. As an example, and not by way oflimitation, computer system 800 may load instructions from storage 808or another source (such as another computer system 800) to the memory804. The processor 806 may then load the instructions from the memory804 to an internal register or internal cache. To execute theinstructions, the processor 806 may retrieve the instructions from theinternal register or internal cache and decode them. During or afterexecution of the instructions, the processor 806 may write one or moreresults (which may be intermediate or final results) to the internalregister or internal cache. The processor 806 may then write one or moreof those results to the memory 804. In particular embodiments, theprocessor 806 executes only instructions in one or more internalregisters or internal caches or in memory 804 (as opposed to storage 808or elsewhere) and operates only on data in one or more internalregisters or internal caches or in memory 804 (as opposed to storage 808or elsewhere). One or more memory buses (which may each include anaddress bus and a data bus) may couple the processor 806 to the memory804. The bus may include one or more memory buses, as described infurther detail below. In particular embodiments, one or more memorymanagement units (MMUs) reside between the processor 806 and memory 804and facilitate accesses to the memory 804 requested by the processor806. In particular embodiments, the memory 804 includes random accessmemory (RAM). This RAM may be volatile memory, where appropriate. Whereappropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM).Moreover, where appropriate, this RAM may be single-ported ormulti-ported RAM. This disclosure contemplates any suitable RAM. Memory804 may include one or more memories 804, where appropriate. Althoughthis disclosure describes and illustrates particular memoryimplementations, this disclosure contemplates any suitable memoryimplementation.

In particular embodiments, the storage 808 includes mass storage fordata or instructions. As an example and not by way of limitation, thestorage 808 may include a hard disk drive (HDD), a floppy disk drive,flash memory, an optical disc, a magneto-optical disc, magnetic tape, ora Universal Serial Bus (USB) drive or a combination of two or more ofthese. The storage 808 may include removable or non-removable (or fixed)media, where appropriate. The storage 808 may be internal or exterior tocomputer system 800, where appropriate. In particular embodiments, thestorage 808 is non-volatile, solid-state memory. In particularembodiments, the storage 808 includes read-only memory (ROM). Whereappropriate, this ROM may be mask-programmed ROM, programmable ROM(PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM),electrically alterable ROM (EAROM), or flash memory or a combination oftwo or more of these. This disclosure contemplates mass storage 808taking any suitable physical form. The storage 808 may include one ormore storage control units facilitating communication between processor806 and storage 808, where appropriate. Where appropriate, the storage808 may include one or more storages 808. Although this disclosuredescribes and illustrates particular storage, this disclosurecontemplates any suitable storage.

In particular embodiments, the I/O Interface 810 includes hardware,software, or both, providing one or more interfaces for communicationbetween computer system 800 and one or more I/O devices. The computersystem 800 may include one or more of these I/O devices, whereappropriate. One or more of these I/O devices may enable communicationbetween a person (i.e., a user) and computer system 800. As an exampleand not by way of limitation, an I/O device may include a keyboard,keypad, microphone, monitor, screen, display panel, mouse, printer,scanner, speaker, still camera, stylus, tablet, touch screen, trackball,video camera, another suitable I/O device or a combination of two ormore of these. An I/O device may include one or more sensors. Whereappropriate, the I/O Interface 810 may include one or more device orsoftware drivers enabling processor 806 to drive one or more of theseI/O devices. The I/O interface 810 may include one or more I/Ointerfaces 810, where appropriate. Although this disclosure describesand illustrates a particular I/O interface, this disclosure contemplatesany suitable I/O interface or combination of I/O interfaces.

In particular embodiments, communication interface 812 includeshardware, software, or both providing one or more interfaces forcommunication (such as, for example, packet-based communication) betweencomputer system 800 and one or more other computer systems 800 or one ormore networks 814. As an example and not by way of limitation,communication interface 812 may include a network interface controller(NIC) or network adapter for communicating with an Ethernet or any otherwire-based network or a wireless NIC (WNIC) or wireless adapter forcommunicating with a wireless network, such as a Wi-Fi network. Thisdisclosure contemplates any suitable network 814 and any suitablecommunication interface 812 for the network 814. As an example and notby way of limitation, the network 814 may include one or more of an adhoc network, a personal area network (PAN), a local area network (LAN),a wide area network (WAN), a metropolitan area network (MAN), or one ormore portions of the Internet or a combination of two or more of these.One or more portions of one or more of these networks may be wired orwireless. As an example, computer system 800 may communicate with awireless PAN (WPAN) (such as, for example, a Bluetooth® WPAN), a WI-FInetwork, a WI-MAX network, a cellular telephone network (such as, forexample, a Global System for Mobile Communications (GSM) network), orany other suitable wireless network or a combination of two or more ofthese. Computer system 800 may include any suitable communicationinterface 812 for any of these networks, where appropriate.Communication interface 812 may include one or more communicationinterfaces 812, where appropriate. Although this disclosure describesand illustrates a particular communication interface implementations,this disclosure contemplates any suitable communication interfaceimplementation.

The computer system 802 may also include a bus. The bus may includehardware, software, or both and may communicatively couple thecomponents of the computer system 800 to each other. As an example andnot by way of limitation, the bus may include an Accelerated GraphicsPort (AGP) or any other graphics bus, an Enhanced Industry StandardArchitecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT)interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBANDinterconnect, a low-PIN-count (LPC) bus, a memory bus, a Micro ChannelArchitecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, aPCI-Express (PCIe) bus, a serial advanced technology attachment (SATA)bus, a Video Electronics Standards Association local bus (VLB), oranother suitable bus or a combination of two or more of these buses. Thebus may include one or more buses, where appropriate. Although thisdisclosure describes and illustrates a particular bus, this disclosurecontemplates any suitable bus or interconnect.

Herein, a computer-readable non-transitory storage medium or media mayinclude one or more semiconductor-based or other types of integratedcircuits (ICs) (e.g., field-programmable gate arrays (FPGAs) orapplication-specific ICs (ASICs)), hard disk drives (HDDs), hybrid harddrives (HHDs), optical discs, optical disc drives (ODDs),magneto-optical discs, magneto-optical drives, floppy diskettes, floppydisk drives (FDDs), magnetic tapes, solid-state drives (SSDs),RAM-drives, SECURE DIGITAL cards or drives, any other suitablecomputer-readable non-transitory storage media, or any suitablecombination of two or more of these, where appropriate. Acomputer-readable non-transitory storage medium may be volatile,non-volatile, or a combination of volatile and non-volatile, whereappropriate.

Herein, “or” is inclusive and not exclusive, unless expressly indicatedotherwise or indicated otherwise by context. Therefore, herein, “A or B”means “A, B, or both,” unless expressly indicated otherwise or indicatedotherwise by context. Moreover, “and” is both joint and several, unlessexpressly indicated otherwise or indicated otherwise by context.Therefore, herein, “A and B” means “A and B, jointly or severally,”unless expressly indicated otherwise or indicated otherwise by context.

The scope of this disclosure encompasses all changes, substitutions,variations, alterations, and modifications to the example embodimentsdescribed or illustrated herein that a person having ordinary skill inthe art would comprehend. The scope of this disclosure is not limited tothe example embodiments described or illustrated herein. Moreover,although this disclosure describes and illustrates respectiveembodiments herein as including particular components, elements,features, functions, operations, or steps, any of these embodiments mayinclude any combination or permutation of any of the components,elements, features, functions, operations, or steps described orillustrated anywhere herein that a person having ordinary skill in theart would comprehend. Furthermore, reference in the appended claims toan apparatus or system or a component of an apparatus or system beingadapted to, arranged to, capable of, configured to, enabled to, operableto, or operative to perform a particular function encompasses thatapparatus, system, component, whether or not it or that particularfunction is activated, turned on, or unlocked, as long as thatapparatus, system, or component is so adapted, arranged, capable,configured, enabled, operable, or operative. Additionally, although thisdisclosure describes or illustrates particular embodiments as providingparticular advantages, particular embodiments may provide none, some, orall of these advantages.

All of the disclosed methods and procedures described in this disclosurecan be implemented using one or more computer programs or components.These components may be provided as a series of computer instructions onany conventional computer readable medium or machine readable medium,including volatile and non-volatile memory, such as RAM, ROM, flashmemory, magnetic or optical disks, optical memory, or other storagemedia. The instructions may be provided as software or firmware, and maybe implemented in whole or in part in hardware components such as ASICs,FPGAs, DSPs, or any other similar devices. The instructions may beconfigured to be executed by one or more processors, which whenexecuting the series of computer instructions, performs or facilitatesthe performance of all or part of the disclosed methods and procedures.

It should be understood that various changes and modifications to theexamples described here will be apparent to those skilled in the art.Such changes and modifications can be made without departing from thespirit and scope of the present subject matter and without diminishingits intended advantages. It is therefore intended that such changes andmodifications be covered by the appended claims.

1. A method comprising: receiving exterior imagery of a structure;determining, with a machine learning model, an exterior surface of thestructure that encloses exterior portions of the structure depictedwithin the exterior imagery; determining, with the machine learningmodel, exterior features of the structure based on the exterior imageryand/or the exterior surface; determining, with the machine learningmodel, an interior model of the structure based on the exterior surfaceand the exterior features; and generating a three-dimensionalrepresentation of interior portions of the structure and exteriorportions of the structure based on the exterior surface and the interiormodel.
 2. The method of claim 1, wherein the exterior features includeat least one of doors, windows, structural support elements, corners,roofs, and/or utility systems of the structure.
 3. The method of claim1, wherein the structure has multiple floors and determining theinterior model comprises determining multiple interior models for themultiple floors.
 4. The method of claim 3, wherein the multiple floorsof the structure are identified based on exterior features of thestructure.
 5. The method of claim 4, wherein the multiple floors areidentified based on multiple levels of windows at multiple heightswithin the structure.
 6. The method of claim 1, wherein the methodfurther comprises, prior to receiving the exterior imagery: receivingtraining data for a plurality of structures, wherein the training dataincludes exterior imagery of the plurality of structures, expectedexterior surfaces for the plurality of structures, expected exteriorfeatures for the plurality of structures, and expected interior modelsfor the plurality of structures; training a first machine learning modelto generate predicted exterior surfaces and predicted exterior featuresof at least a subset of the plurality of structures based at least on(i) exterior imagery of the subset of the plurality of structures and(ii) expected exterior surfaces of the subset of the plurality ofstructures; and training the first machine learning model to predictinterior models for at least the subset of the plurality of structuresbased at least on the predicted exterior surfaces and the predictedexterior features.
 7. The method of claim 6, wherein the method furthercomprises, prior to receiving the training data: receiving a pluralityof architectural plans for the plurality of structures; and generating,with a second machine learning model, the expected interior models forthe plurality of structures based on the plurality of architecturalplans.
 8. The method of claim 6, wherein training the first machinelearning model to predict exterior contours and exterior featurescomprises: receiving first exterior imagery of a first structure fromthe plurality of structures; predicting, with the first machine learningmodel, a first exterior surface of the first structure and firstexterior features based on the first exterior imagery; detecting one ormore differences (i) between the first exterior surface of the firststructure and an expected exterior surface of the first structure and/or(ii) between the first exterior features and expected exterior featuresof the first structure; and adjusting one or more parameters of thefirst machine learning model based on the one or more differences. 9.The method of claim 6, wherein training the first machine learning modelto predict the interior models comprises: receiving an exterior contourand exterior features of a first structure from the plurality ofstructures; predicting, with the first machine learning model, a firstinterior model of the first structure based on the exterior contour ofthe first structure; detecting one or more differences between the firstinterior model of the first structure and an expected interior model ofthe first structure; and adjusting one or more parameters of the firstmachine learning model based on the one or more differences.
 10. Themethod of claim 1, wherein the structure includes at least one of: abuilding, a vehicle, an infrastructure component, a ship, a spacecraft,an aircraft, a tank, and/or an appliance.
 11. A method comprising:receiving training data for a plurality of structures, wherein thetraining data includes exterior imagery of the plurality of structures,expected exterior surfaces for the plurality of structures, expectedexterior features for the plurality of structures, and expected interiormodels for the plurality of structures; training a first machinelearning model to generate predicted exterior surfaces and predictedexterior features for at least a subset of the plurality of structuresbased at least on (i) exterior imagery of the subset of the plurality ofstructures and (ii) expected exterior surfaces of the subset of theplurality of structures; training the first machine learning model topredict interior models for at least the subset of the plurality ofstructures based at least on the predicted exterior surfaces and thepredicted exterior features; and deploying the first machine learningmodel to predict exterior contours and interior models for additionalstructures separate from the plurality of structures.
 12. The method ofclaim 11, further comprising, prior to receiving the training data:receiving a plurality of architectural plans for the plurality ofstructures; and generating, with a second machine learning model, theexpected interior models for the plurality of structures based on theplurality of architectural plans.
 13. The method of claim 11, whereintraining the first machine learning model to predict exterior contoursand exterior features comprises: receiving first exterior imagery of afirst structure from the plurality of structures; predicting, with thefirst machine learning model, a first exterior surface of the firststructure and first exterior features based on the first exteriorimagery; detecting one or more differences (i) between the firstexterior surface of the first structure and an expected exterior surfaceof the first structure and/or (ii) between the first exterior featuresand expected exterior features of the first structure; and adjusting oneor more parameters of the first machine learning model based on the oneor more differences.
 14. The method of claim 11, wherein training thefirst machine learning model to predict the interior models comprises:receiving an exterior surface and exterior features of a first structurefrom the plurality of structures; predicting, with the first machinelearning model, a first interior model of the first structure based onthe exterior surface of the first structure; detecting one or moredifferences between the first interior model of the first structure andan expected interior model of the first structure; and adjusting one ormore parameters of the first machine learning model based on the one ormore differences.
 15. The method of claim 14, wherein the exteriorsurface of the first structure is one of an expected exterior surface ofthe first structure included within the training data and/or a predictedexterior surface of the first structure generated by the first machinelearning model, and wherein the exterior features of the first structureare one of expected exterior features of the first structure includedwithin the training data and/or predicted exterior features of the firststructure generated by the first machine learning model.
 16. The methodof claim 11, wherein training the first machine learning model topredict the interior models comprises training the first machinelearning model to generate interior models that comply with at least oneof (i) spatial constraints of the exterior surfaces and/or (ii) commonconstruction methods and structural design requirements.
 17. The methodof claim 16, wherein the at least one of (i) the spatial constraints ofthe exterior surfaces and/or (ii) the common construction methods andstructural design requirements are represented within an objectivefunction for the first machine learning model.
 18. The method of claim11, wherein deploying the first machine learning model furthercomprises: receiving exterior imagery of a structure; determining, witha machine learning model, an exterior surface, exterior features, and aninterior model of the structure; and generating a three-dimensionalrepresentation of interior portions of the structure and exteriorportions of the structure based on the exterior surface and the interiormodel.
 19. The method of claim 18, wherein determining the exteriorcontours, exterior features, and an interior model of the structurefurther comprises: determining, with a machine learning model, anexterior surface of the structure that encloses exterior portions of thestructure depicted within the exterior imagery; determining, with themachine learning model, exterior features of the structure based on theexterior imaging and/or the exterior surface; and determining, with themachine learning model, an interior model of the structure based on theexterior contours and the exterior features.
 20. The method of claim 13,wherein the exterior features include at least one of doors, windows,structural support elements, corners, and/or utility systems of thestructure.